<html>
  <head>
    <meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>神经网络 | JIANG-HS</title>
<link rel="shortcut icon" href="https://JIANG-HS.github.io/favicon.ico?v=1600693149942">
<link href="https://cdn.jsdelivr.net/npm/remixicon@2.3.0/fonts/remixicon.css" rel="stylesheet">
<link rel="stylesheet" href="https://JIANG-HS.github.io/styles/main.css">
<link rel="alternate" type="application/atom+xml" title="神经网络 | JIANG-HS - Atom Feed" href="https://JIANG-HS.github.io/atom.xml">
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Droid+Serif:400,700">


<script async src="https://www.googletagmanager.com/gtag/js?id=UA-xxxxxxxxx-x"></script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());

  gtag('config', 'UA-xxxxxxxxx-x');
</script>


    <meta name="description" content="一、人工神经网络
1.1 介绍
 人工神经网络( Artificial Neural Network, ANN)是20世纪80年代以来人工智能领域兴起的研究热点。它从信息处理角度对人脑神经元网络进行抽象，构建某种简单模型，按不同的连接方式组..." />
    <meta name="keywords" content="神经网络,BP神经网络,人工智能" />
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.10.0/katex.min.css">
    <script src="https://cdn.bootcss.com/highlight.js/9.12.0/highlight.min.js"></script>
  </head>
  <body>
    <div class="main">
      <div class="main-content">
        <div class="site-header">
  <a href="https://JIANG-HS.github.io">
  <img class="avatar" src="https://JIANG-HS.github.io/images/avatar.png?v=1600693149942" alt="">
  </a>
  <h1 class="site-title">
    JIANG-HS
  </h1>
  <p class="site-description">
    姜还是老的辣
  </p>
  <div class="menu-container">
    
      
        <a href="/" class="menu">
          首页
        </a>
      
    
      
        <a href="/archives" class="menu">
          归档
        </a>
      
    
      
        <a href="/tags" class="menu">
          标签
        </a>
      
    
      
        <a href="/post/about" class="menu">
          关于
        </a>
      
    
  </div>
  <div class="social-container">
    
      
    
      
    
      
    
      
    
      
    
  </div>
</div>

        <div class="post-detail">
          <article class="post">
            <h2 class="post-title">
              神经网络
            </h2>
            <div class="post-info">
              <span>
                2020-07-28
              </span>
              <span>
                9 min read
              </span>
              
                <a href="https://JIANG-HS.github.io/tag/vQdk_Z6zT/" class="post-tag">
                  # 神经网络
                </a>
              
                <a href="https://JIANG-HS.github.io/tag/SQTtm2VX_7/" class="post-tag">
                  # BP神经网络
                </a>
              
                <a href="https://JIANG-HS.github.io/tag/hPqZti1l1/" class="post-tag">
                  # 人工智能
                </a>
              
            </div>
            
            <div class="post-content-wrapper">
              <div class="post-content">
                <h1 id="一-人工神经网络">一、人工神经网络</h1>
<h2 id="11-介绍">1.1 介绍</h2>
<p> 人工神经网络( Artificial Neural Network, ANN)是20世纪80年代以来人工智能领域兴起的研究热点。它从信息处理角度对人脑神经元网络进行抽象，构建某种简单模型，按不同的连接方式组成不同的网络。在工程与学术界常将人工神经网络简称为<strong>神经网络（NN）</strong>。</p>
<p> 神经网络是一种运算模型，由大量的节点(或称神经元)之间相互连接构成。每个节点代表一种特定的输出函数，称为<strong>激励函数</strong>或者<strong>激活函数(activation function)</strong>。每两个节点间的连接都代表一个对于通过该连接信号的加权值，称之为<strong>权重</strong>，这相当于人工神经网络的记忆。</p>
<p> 神经网络的输出根据网络的连接方式、权重值和激活函数的不同而不同。而网络自身通常都是对自然界某种算法或者函数的逼近，也可能是对一种逻辑策略的表达。简而言之，搭建人工神经网络利用函数拟合的性质体现自然规律。</p>
<p> 目前，人工神经网络在模式识别、智能机器人、自动控制、预测估计、生物、医学和经济等领域应用广泛。</p>
<h2 id="12-神经网络的特点">1.2 神经网络的特点</h2>
<ul>
<li>非线性：非线性关系是自然界的普遍特性。大脑的智慧就是一种非线性现象。人工神经元处于激活或抑制两种不同的状态，这种行为在数学上表现为一种非线性关系。具有阈值的神经元构成的网络具有更好的性能，可以提高容错性和存储容量。</li>
<li>非局限性：一个神经网络通常由多个神经元广泛连接而成。一个系统的整体行为不仅取决于单个神经元的特征，而且由单元之间的相互作用、相互连接所决定，通过单元之间的大量连接模拟大脑的非局限性。联想记忆是非局限性的典型例子。</li>
<li>非常定性：人工神经网络具有自适应、自组织和自学习能力。神经网络不但处理的信息可以有各种变化，而且在处理信息的同时，非线性动力系统本身也在不断变化，经常采用迭代过程描写动力系统的演化过程。</li>
<li>非凸性：一个系统的演化方向，在一定条件下将取决于某个特定的状态函数。例如能量函数，它的极值表示为系统比较稳定的状态。非凸性是指这种函数有多个极值，因此系统具有多个较稳定的平衡态，这将导致系统演化的多样性。</li>
</ul>
<h2 id="13-激活函数">1.3 激活函数</h2>
<p> 激活函数又称非线性映射，顾名思义，激活函数的引入为的是<strong>增加整个网络的表达能力(即非线性)</strong>，否则，若干线性操作层的堆叠仍然只能起到线性映射的作用，无法形成复杂的函数。下面将介绍几种常见的激活函数。<br>
 激活函数应该具有的性质如下：</p>
<ul>
<li>非线性：线性激活对于深层神经网络没有作用，因为其作用以后仍然是输入的各种线性变换。</li>
<li>连续可微：梯度下降法的要求。</li>
<li>范围最好不饱和，当有饱和的区间段时，若系统优化进入到该段，梯度近似为0，网络的学习就会停止。</li>
<li>单调性：当激活函数是单调时，单层神经网络的误差函数是凸的，好优化。</li>
<li>在原点处近似线性，这样当权值初始化为接近0的随机值时，网络可以学习得较快，不用调节网络的初始值。</li>
</ul>
<p>通常使用的激活函数有sigmoid、tanh和relu 函数。<br>
①Sigmoid函数：也称S型生长曲线，在神将网络研究初期曾一度非常受欢迎，其输出在0和1之间，可以将输入数据压缩化，增加模型的稳定性。但由于其对数据的压缩，会造成数据的梯度降低甚至消失。函数图像如下：<br>
<img src="https://JIANG-HS.github.io/post-images/1590804552355.png" alt="" loading="lazy"><br>
<img src="https://JIANG-HS.github.io/post-images/1590804559256.jpg" alt="" loading="lazy"><br>
②Tanh函数：是一个双曲正切函数，它比sigmoid函数收敛速度更快。相比sigmoid函数，tanh函数的输出以0为中心。但是还是没有改变梯度降低或者消失的问题。函数图像如下：<br>
<img src="https://JIANG-HS.github.io/post-images/1590804750456.png" alt="" loading="lazy"><br>
<img src="https://JIANG-HS.github.io/post-images/1590804754119.jpg" alt="" loading="lazy"><br>
③ReLU函数(The Rectified Linear Unit)：relu函数在梯度下降中能够快速收敛，它的优点在于没有expensive的操作（比如指数），relu函数可以更简单地实现且有效缓解了梯度消失的问题。但是随着训练的继续，可能会出现神经元死亡，权重无法更新的问题。函数图像如下：<br>
<img src="https://JIANG-HS.github.io/post-images/1590805753654.png" alt="" loading="lazy"><br>
<img src="https://JIANG-HS.github.io/post-images/1590805765708.png" alt="" loading="lazy"></p>
<h1 id="二-由单层神经网络到多层神经网络">二、由单层神经网络到多层神经网络</h1>
<h2 id="21-单层神经网络">2.1 单层神经网络</h2>
<p><img src="https://JIANG-HS.github.io/post-images/1596089635211.jpg" alt="" loading="lazy"><br>
上图是一个最简单的单层神经网络，包括输入、权重和输出。这个神经元<span class="katex"><span class="katex-mathml"><math><semantics><mrow><msub><mi>a</mi><mn>1</mn></msub></mrow><annotation encoding="application/x-tex">a_{1}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.58056em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>、<span class="katex"><span class="katex-mathml"><math><semantics><mrow><msub><mi>a</mi><mn>2</mn></msub></mrow><annotation encoding="application/x-tex">a_{2}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.58056em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>、<span class="katex"><span class="katex-mathml"><math><semantics><mrow><msub><mi>a</mi><mn>3</mn></msub></mrow><annotation encoding="application/x-tex">a_{3}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.58056em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">3</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>作是输入，<span class="katex"><span class="katex-mathml"><math><semantics><mrow><msub><mi>w</mi><mn>1</mn></msub></mrow><annotation encoding="application/x-tex">w_{1}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.58056em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathdefault" style="margin-right:0.02691em;">w</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:-0.02691em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>、<span class="katex"><span class="katex-mathml"><math><semantics><mrow><msub><mi>w</mi><mn>2</mn></msub></mrow><annotation encoding="application/x-tex">w_{2}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.58056em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathdefault" style="margin-right:0.02691em;">w</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:-0.02691em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>、<span class="katex"><span class="katex-mathml"><math><semantics><mrow><msub><mi>w</mi><mn>3</mn></msub></mrow><annotation encoding="application/x-tex">w_{3}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.58056em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathdefault" style="margin-right:0.02691em;">w</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:-0.02691em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">3</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>是权重，输入节点后，经过激活函数，得到输出z。其矩阵计算公式为:</p>
<p class='katex-block'><span class="katex-display"><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>g</mi><mo>(</mo><mi>W</mi><mo>∗</mo><mi>a</mi><mo>)</mo><mo>=</mo><mi>z</mi></mrow><annotation encoding="application/x-tex">g(W*a)=z
</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathdefault" style="margin-right:0.03588em;">g</span><span class="mopen">(</span><span class="mord mathdefault" style="margin-right:0.13889em;">W</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathdefault">a</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:0.43056em;vertical-align:0em;"></span><span class="mord mathdefault" style="margin-right:0.04398em;">z</span></span></span></span></span></p>
<p>单层神经网络具有模型清晰、结构简单、计算量小等优点。但是随着研究的深入，我们发现它不能很好地处理非线性的问题。</p>
<h2 id="22-双层神经网络">2.2 双层神经网络</h2>
<p>两层神经网络除了包含一个输入层和一个输出层以外，还增加了一个中间层（隐层），此时有中间层和输出层两个计算层。<br>
<img src="https://JIANG-HS.github.io/post-images/1596090769865.jpg" alt="" loading="lazy"><br>
图中的<span class="katex"><span class="katex-mathml"><math><semantics><mrow><msubsup><mi>a</mi><mn>1</mn><mrow><mo>(</mo><mn>2</mn><mo>)</mo></mrow></msubsup></mrow><annotation encoding="application/x-tex">a_{1}^{(2)}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.311108em;vertical-align:-0.26630799999999993em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.0448em;"><span style="top:-2.433692em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">1</span></span></span></span><span style="top:-3.2198em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">2</span><span class="mclose mtight">)</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.26630799999999993em;"><span></span></span></span></span></span></span></span></span></span>和<span class="katex"><span class="katex-mathml"><math><semantics><mrow><msubsup><mi>a</mi><mn>2</mn><mrow><mo>(</mo><mn>2</mn><mo>)</mo></mrow></msubsup></mrow><annotation encoding="application/x-tex">a_{2}^{(2)}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.311108em;vertical-align:-0.26630799999999993em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.0448em;"><span style="top:-2.433692em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span><span style="top:-3.2198em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">2</span><span class="mclose mtight">)</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.26630799999999993em;"><span></span></span></span></span></span></span></span></span></span>为该神经网络的隐层。最左边为输入层<span class="katex"><span class="katex-mathml"><math><semantics><mrow><msup><mi>a</mi><mrow><mo>(</mo><mn>1</mn><mo>)</mo></mrow></msup></mrow><annotation encoding="application/x-tex">a^{(1)}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.8879999999999999em;vertical-align:0em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8879999999999999em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">1</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span></span></span></span>，最用边为输出层，两组权重分别为<span class="katex"><span class="katex-mathml"><math><semantics><mrow><msup><mi>W</mi><mrow><mo>(</mo><mn>1</mn><mo>)</mo></mrow></msup></mrow><annotation encoding="application/x-tex">W^{(1)}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.8879999999999999em;vertical-align:0em;"></span><span class="mord"><span class="mord mathdefault" style="margin-right:0.13889em;">W</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8879999999999999em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">1</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span></span></span></span>和<span class="katex"><span class="katex-mathml"><math><semantics><mrow><msup><mi>W</mi><mrow><mo>(</mo><mn>2</mn><mo>)</mo></mrow></msup></mrow><annotation encoding="application/x-tex">W^{(2)}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.8879999999999999em;vertical-align:0em;"></span><span class="mord"><span class="mord mathdefault" style="margin-right:0.13889em;">W</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8879999999999999em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">2</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span></span></span></span>。<br>
其矩阵计算公式变化为：</p>
<p class='katex-block'><span class="katex-display"><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>g</mi><mo>(</mo><msup><mi>W</mi><mrow><mo>(</mo><mn>1</mn><mo>)</mo></mrow></msup><mo>∗</mo><msup><mi>a</mi><mrow><mo>(</mo><mn>1</mn><mo>)</mo></mrow></msup><mo>)</mo><mo>=</mo><msup><mi>a</mi><mrow><mo>(</mo><mn>2</mn><mo>)</mo></mrow></msup></mrow><annotation encoding="application/x-tex">g(W^{(1)}*a^{(1)})=a^{(2)}
</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.188em;vertical-align:-0.25em;"></span><span class="mord mathdefault" style="margin-right:0.03588em;">g</span><span class="mopen">(</span><span class="mord"><span class="mord mathdefault" style="margin-right:0.13889em;">W</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.938em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">1</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1.188em;vertical-align:-0.25em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.938em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">1</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:0.938em;vertical-align:0em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.938em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">2</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span></span></span></span></span></p>
<p class='katex-block'><span class="katex-display"><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>g</mi><mo>(</mo><msup><mi>W</mi><mrow><mo>(</mo><mn>2</mn><mo>)</mo></mrow></msup><mo>∗</mo><msup><mi>a</mi><mrow><mo>(</mo><mn>2</mn><mo>)</mo></mrow></msup><mo>)</mo><mo>=</mo><mi>z</mi></mrow><annotation encoding="application/x-tex">g(W^{(2)}*a^{(2)})=z
</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.188em;vertical-align:-0.25em;"></span><span class="mord mathdefault" style="margin-right:0.03588em;">g</span><span class="mopen">(</span><span class="mord"><span class="mord mathdefault" style="margin-right:0.13889em;">W</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.938em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">2</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1.188em;vertical-align:-0.25em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.938em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">2</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:0.43056em;vertical-align:0em;"></span><span class="mord mathdefault" style="margin-right:0.04398em;">z</span></span></span></span></span></p>
<p><strong>与单层神经网络不通，两层神经网络可以无限逼近任意连续函数。也就是说，面对复杂的非线性分类任务，两层神经网络可以很好地分类。</strong></p>
<h2 id="23-多层神经网络">2.3 多层神经网络</h2>
<p>在两层神经网络的基础上再增加一个或者更多个隐层，就构成了多层的神经网络，此时计算层的数量为三个或更多。<br>
<img src="https://JIANG-HS.github.io/post-images/1596093858412.jpg" alt="" loading="lazy"><br>
上图是一个带有两个隐层的多层神经网络。其矩阵运算公式为：</p>
<p class='katex-block'><span class="katex-display"><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>g</mi><mo>(</mo><msup><mi>W</mi><mrow><mo>(</mo><mn>1</mn><mo>)</mo></mrow></msup><mo>∗</mo><msup><mi>a</mi><mrow><mo>(</mo><mn>1</mn><mo>)</mo></mrow></msup><mo>)</mo><mo>=</mo><msup><mi>a</mi><mrow><mo>(</mo><mn>2</mn><mo>)</mo></mrow></msup></mrow><annotation encoding="application/x-tex">g(W^{(1)}*a^{(1)})=a^{(2)}
</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.188em;vertical-align:-0.25em;"></span><span class="mord mathdefault" style="margin-right:0.03588em;">g</span><span class="mopen">(</span><span class="mord"><span class="mord mathdefault" style="margin-right:0.13889em;">W</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.938em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">1</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1.188em;vertical-align:-0.25em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.938em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">1</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:0.938em;vertical-align:0em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.938em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">2</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span></span></span></span></span></p>
<p class='katex-block'><span class="katex-display"><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>g</mi><mo>(</mo><msup><mi>W</mi><mrow><mo>(</mo><mn>2</mn><mo>)</mo></mrow></msup><mo>∗</mo><msup><mi>a</mi><mrow><mo>(</mo><mn>2</mn><mo>)</mo></mrow></msup><mo>)</mo><mo>=</mo><msup><mi>a</mi><mrow><mo>(</mo><mn>3</mn><mo>)</mo></mrow></msup></mrow><annotation encoding="application/x-tex">g(W^{(2)}*a^{(2)})=a^{(3)}
</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.188em;vertical-align:-0.25em;"></span><span class="mord mathdefault" style="margin-right:0.03588em;">g</span><span class="mopen">(</span><span class="mord"><span class="mord mathdefault" style="margin-right:0.13889em;">W</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.938em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">2</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1.188em;vertical-align:-0.25em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.938em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">2</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:0.938em;vertical-align:0em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.938em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">3</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span></span></span></span></span></p>
<p class='katex-block'><span class="katex-display"><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>g</mi><mo>(</mo><msup><mi>W</mi><mrow><mo>(</mo><mn>3</mn><mo>)</mo></mrow></msup><mo>∗</mo><msup><mi>a</mi><mrow><mo>(</mo><mn>3</mn><mo>)</mo></mrow></msup><mo>)</mo><mo>=</mo><mi>z</mi></mrow><annotation encoding="application/x-tex">g(W^{(3)}*a^{(3)})=z
</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.188em;vertical-align:-0.25em;"></span><span class="mord mathdefault" style="margin-right:0.03588em;">g</span><span class="mopen">(</span><span class="mord"><span class="mord mathdefault" style="margin-right:0.13889em;">W</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.938em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">3</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1.188em;vertical-align:-0.25em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.938em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">3</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:0.43056em;vertical-align:0em;"></span><span class="mord mathdefault" style="margin-right:0.04398em;">z</span></span></span></span></span></p>
<p> 与两层神经网络不同，随着网络的层数增加，每一层对于前一层次的抽象表示更深入。在神经网络中，每一层神经元学习到的是前一层神经元值的更抽象的表示。例如第一个隐藏层学习到的是“边缘”的特征，第二个隐藏层学习到的是由“边缘”组成的“形状”的特征，第三个隐藏层学习到的是由“形状”组成的“图案”的特征，最后的隐藏层学习到的是由“图案”组成的“目标”的特征。</p>
<h1 id="二-bp神经网络">二、BP神经网络</h1>
<p>BP神经网络是一种非线性多层前向反馈网络，也就是多了一个反向传播的过程。基本思路就是，模型每进行一次前向传播之后，计算输出层与目标函数之间的误差，再将结果代入激活函数的导数计算之后，返回给离输出层最近的隐层，再计算当前隐层与上一层之间的误差，然后逐渐往回传播，直到第一个隐层为止。进行一次反向传播之后，还需要对权重参数进行更新。</p>
<h1 id="二-简单代码实现">二、简单代码实现</h1>
<pre><code>import numpy as np  

#定义激活函数，这里使用到的是Sigmoid函数
def nonlin(x,deriv=False):  
    if(deriv==True):  #定义Sigmoid的导数
        return x*(1-x)  
    return 1/(1+np.exp(-x)) #Sigomid函数

#定义输入数据      
X = np.array([[0,0,1],  
            [0,1,1],  
            [1,0,1],  
            [1,1,1]]) 
#print (X.shape) 

#目标比对模型                  
y = np.array([[0],  
            [1],  
            [1],  
            [0]])  
#print (y.shape)
np.random.seed(1)  

# randomly initialize our weights with mean 0  
w0 = 2*np.random.random((3,4)) - 1  
w1 = 2*np.random.random((4,1)) - 1
#print (w0)
#print (w1)  
#print (w0.shape)
#print (w1.shape)

for j in range(60000):
    #前向传播，l0为输入层，l1为隐层，l2为输出层
    l0 = X  
    l1 = nonlin(np.dot(l0,w0))  #矩阵运算
    l2 = nonlin(np.dot(l1,w1))
    
    l2_error = y - l2  
    #打印误差值
    if (j% 10000) == 0:  
        print (&quot;Error:&quot; + str(np.mean(np.abs(l2_error))))  

    #反向传播          
    l2_delta = l2_error * nonlin(l2,deriv=True)       
    l1_error = l2_delta.dot(w1.T)  
    l1_delta = l1_error * nonlin(l1,deriv=True)  

    #更新权重
    alpha = 0.5 #学习率
    w1 += alpha * l1.T.dot(l2_delta)  
    w0 += alpha * l0.T.dot(l1_delta)
</code></pre>

              </div>
              <div class="toc-container">
                <ul class="markdownIt-TOC">
<li><a href="#%E4%B8%80-%E4%BA%BA%E5%B7%A5%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C">一、人工神经网络</a>
<ul>
<li><a href="#11-%E4%BB%8B%E7%BB%8D">1.1 介绍</a></li>
<li><a href="#12-%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E7%9A%84%E7%89%B9%E7%82%B9">1.2 神经网络的特点</a></li>
<li><a href="#13-%E6%BF%80%E6%B4%BB%E5%87%BD%E6%95%B0">1.3 激活函数</a></li>
</ul>
</li>
<li><a href="#%E4%BA%8C-%E7%94%B1%E5%8D%95%E5%B1%82%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E5%88%B0%E5%A4%9A%E5%B1%82%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C">二、由单层神经网络到多层神经网络</a>
<ul>
<li><a href="#21-%E5%8D%95%E5%B1%82%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C">2.1 单层神经网络</a></li>
<li><a href="#22-%E5%8F%8C%E5%B1%82%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C">2.2 双层神经网络</a></li>
<li><a href="#23-%E5%A4%9A%E5%B1%82%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C">2.3 多层神经网络</a></li>
</ul>
</li>
<li><a href="#%E4%BA%8C-bp%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C">二、BP神经网络</a></li>
<li><a href="#%E4%BA%8C-%E7%AE%80%E5%8D%95%E4%BB%A3%E7%A0%81%E5%AE%9E%E7%8E%B0">二、简单代码实现</a></li>
</ul>

              </div>
            </div>
          </article>
        </div>

        
          <div class="next-post">
            <div class="next">下一篇</div>
            <a href="https://JIANG-HS.github.io/post/2D2skmyaX/">
              <h3 class="post-title">
                协同过滤算法
              </h3>
            </a>
          </div>
        

        
          
            <link rel="stylesheet" href="https://unpkg.com/gitalk/dist/gitalk.css">
<script src="https://unpkg.com/gitalk/dist/gitalk.min.js"></script>

<div id="gitalk-container"></div>

<script>

  var gitalk = new Gitalk({
    clientID: '',
    clientSecret: '',
    repo: '',
    owner: '',
    admin: [''],
    id: (location.pathname).substring(0, 49),      // Ensure uniqueness and length less than 50
    distractionFreeMode: false  // Facebook-like distraction free mode
  })

  gitalk.render('gitalk-container')

</script>

          

          
        

        <div class="site-footer">
  Powered by <a href="https://github.com/getgridea/gridea" target="_blank">Gridea</a>
  <a class="rss" href="https://JIANG-HS.github.io/atom.xml" target="_blank">
    <i class="ri-rss-line"></i> RSS
  </a>
</div>

      </div>
    </div>

    <script>
      hljs.initHighlightingOnLoad()

      let mainNavLinks = document.querySelectorAll(".markdownIt-TOC a");

      // This should probably be throttled.
      // Especially because it triggers during smooth scrolling.
      // https://lodash.com/docs/4.17.10#throttle
      // You could do like...
      // window.addEventListener("scroll", () => {
      //    _.throttle(doThatStuff, 100);
      // });
      // Only not doing it here to keep this Pen dependency-free.

      window.addEventListener("scroll", event => {
        let fromTop = window.scrollY;

        mainNavLinks.forEach((link, index) => {
          let section = document.getElementById(decodeURI(link.hash).substring(1));
          let nextSection = null
          if (mainNavLinks[index + 1]) {
            nextSection = document.getElementById(decodeURI(mainNavLinks[index + 1].hash).substring(1));
          }
          if (section.offsetTop <= fromTop) {
            if (nextSection) {
              if (nextSection.offsetTop > fromTop) {
                link.classList.add("current");
              } else {
                link.classList.remove("current");    
              }
            } else {
              link.classList.add("current");
            }
          } else {
            link.classList.remove("current");
          }
        });
      });

    </script>
  </body>
</html>
